In the past, the performance of computer software system, including operating system (OS) and its applications, has been a concern of software vendors and their customers, since system performance relates directly to the service quality and success of software product sales. In general there are two angles to view the issue of performance:
The first angle views the “consequence” of application software, or the “user experience” as performance. For example, software code is inserted into applications to measure the “response time” from the return key to the response appearing on computer screen. The performance measurement code may be combined with application functions to form a complicated Performance Monitor, such as Couture's patent US20080235075 “Enterprise application performance monitors”.
The second angle views the “cause” that impacts the performance of running software. The impacting root cause is, of course, the hardware resources such as CPU, memory, IO and network. In the past there is no way to measure performance from hardware, hence the second best way can only utilize the OS performance commands to measure performance of the resources like threads, processes, etc. Lately, new hardware monitoring base tool emerges, such as x86-based hardware register capable of monitoring CPU, called Performance Monitor Unit (PMU). There are quite a few PMU patents: e.g. Davidson's U.S. Pat. No. 6,718,403 “Hierarchical selection of direct and indirect counting events in a PMU”, Mericas's U.S. Pat. No. 7,500,138 “Simplified event selection for a PMU” and Mericas's US20060167658 “Method and product of PMU for sampling all performance events generated by a processor”. There are also non-PMU patents that are hardware and performance monitoring based, such as Fowles's US20060277395 “Processor performance monitoring”; Kosche's US20080177756 “Method and apparatus for synthesizing hardware counters from performance sampling”, and Hunter's US20080294944 “Processor bus for performance monitoring with digests”.
Two kinds of registers help monitoring performance: performance control register (PMC) and performance data register (PMD). PMC monitors CPU at regular time intervals, and saves monitored data into PMD. PMC is event-based or time-based to consolidate the collected event information and report the monitoring results to higher software layers. Advantages of hardware monitoring are: (1) in the past, certain software-based monitors require to continuously change the source code of the monitored software to reach its goal, e.g. optimizing a Java virtual machine routine. On the contrary, hardware-based monitoring requires no source code. (2) The performance of monitored software is not affected by monitoring activities. (3) Very low-level kernel code can be monitored. Most importantly, (4) Capability to monitor cache is nowhere found in any other monitoring means. One example is TLB miss and hit, where TLB stands for “translation look-aside buffer”, the CPU cache hardware for memory management to improve the speed of virtual H physical address translation. TLB takes virtual address as search keyword. The search result is physical address: if the needed address can be found in TLB, it is called TLB hit. Otherwise, it is called TLB miss. Frequent TLB misses indicate the performance of resource is degraded. The details of PMU are described in the document written by Eranian, Hewlett Packard Company, “The perfmon2 interface specification”.
There are not very many PMU applications. Moreover, there is no existing method to utilize PMU and combine OS performance commands to measure the performance of “virtualized systems”. Here is the explanation of virtualized systems:
The technology of OS virtualization leads to the result of a physical machine is capable of running multiple “guest” OSs (or Virtual Machine, briefly VM, or simply “guest”). The VMs run on VMM (virtual machine monitor, or hypervisor), and Hypervisor runs directly on physical machine. In most cases, the monitored physical machines are servers (or host) in the data center. There are different OS virtualization techniques, such as para-virtualization or full virtualization. In general, a host has at least two spaces to be monitored: one is the space where VMs are running, called (1) guest space; the other is the space where a hypervisor is running, called (2) kernel space. Some virtualization technique, such as Linux KVM, keeps the use of the original (3) user space. The above explains the internals of a host. Multiple such hosts form a cluster, with a lead host called master, and several subordinate hosts called slaves. Multiple clusters become a network system, being distributed, centralized, or mixed. This explains “virtualized systems” of the disclosed invention, a network system covering large geographical areas.
Tang's patent US20110296411 “Kernel bus system to build Virtual Machine Monitor and the performance service framework & Method therefore” describes how virtualized systems analyze performance data, perform VM migration, and mediate workload. The patent however does not describe how virtualized systems acquire performance data.